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Top-down inputs from prefrontal cortex impact on sen- 
sory neurons [1,2], enhancing their selectivity to 
attended stimuli, while sensory processing of distractors 
is suppressed [1,3,4]. However, what are the neuro-com- 
putational mechanisms that identify the behaviorally 
relevant information that is worth to bias? Previous stu- 
dies linked selective attention to learned value [5], sug- 
gesting that attentional selection relies on internal 
mechanisms that track the relevance of sensory informa- 
tion. Consistent with this view, we recently introduced a 
reinforcement learning (RL) approach for the deploy- 
ment of selective attention [6]. We tested model-free 
and model-based versions of RL to identify the mechan- 
isms that most accurately predict behavior: whereas 
model-based prioritizes attentional selection to task fea- 
tures that are systematically associated with reward, 
model-free considers all available features. Our results 
proved that the optimal task-set representation signifi- 
cantly improved predictive power, suggesting that mon- 
keys benefited from model-based mechanisms [6]. Yet, 
model-based presents two important limitations: i) it is 
unable to adapt to changes in the association of reward 
with sensory features that are not included in the 
learned model, and ii) it excludes the mechanism of 
learning by which subjects derive the proper task-set 
representation. The question remains then of how a 
prioritized task set can be learned to exploit the benefits 
of employing a model. 

With the aim to workaround model-based limitations 
and shed light on the underlying mechanisms that make 
model-based benefits possible, we propose here the 
"adaptive learning" mechanism for flexible task-set 
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representation. The mechanism is dynamically tuned 
according to the statistics of association between sensory 
features and reward outcome, flexibly adjusting its 
regime of operation between model-free and model- 
based systems. To test the adaptive learning mechanism, 
we developed it in a RL model and extended our pre- 
vious analysis of monkey behavior to both, cued [5] and 
uncued [6], versions of the same attention task. This RL 
model was able to transition from a naive starting point 
to an optimal task-set representation, and to flexibly 
adapt among optimal task-set representations upon 
changes in reward contingencies. The model achieved 
so by tracking a separate learning rate a for each stimu- 
lus feature in the environment. If the selection of a sti- 
mulus feature was systematically correlated with a 
particular outcome (reward or no-reward), a increased. 
In contrast, when a feature was unable to consistently 
predict reward, a decayed. Thus, a changed dynami- 
cally, and over a large number of trials any a associated 
with a non-predictive feature went to zero, effectively 
eliminating it from the task set, which tuned perfor- 
mance towards that of the model-based system. 

The adaptive learning mechanism introduced here 
represents a step further in our understanding of the ori- 
gins of selective attention. Notably, our results prove that 
the adaptive learning is an optimal mechanistic candidate 
to support arbitrary prioritized model-based formation 
under generic conditions of covert attentional selection, 
and regardless of whether attentional selection operates 
on cued or uncued tasks. 



Acknowledgements 

This work is supported by the Canadian Institutes of Health Research and 
Natural Sciences and Engineering Research Council of Canada. 



o 



© 2014 Ardid et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons 
DiftlUlml C 'an-hr-al Attribution License (http://creativecommons.Org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in 
DIUIYKsU \_fcri llldl an y medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http:// 

creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. 



Ardid et al. BMC Neuroscience 2014, 15(Suppl 1):P8 
http://www.biomedcentral.com/1471-2202/15/S1/P8 



Page 2 of 2 



Published: 21 July 2014 
References 

1 . Ardid S, Wang X-J, Compte A: An integrated microcircuit model of 
attentional processing in the neocortex. J Neurosci 2007, 
27(32):8486-8495. 

2. Gregoriou GG, Gotts SJ, Zhou H, Desimone R: High-frequency, long-range 
coupling between prefrontal and visual cortex during attention. Science 
2009, 324(5931 ):1 207-1 210. 

3. Reynolds JH, Chelazzi L: Attentional modulation of visual processing. 
Annu Rev Neurosci 2004, 27:61 1-647. 

4. Maunsell JH, Treue S: Feature-based attention in visual cortex. Trends 
Neurosci 2006, 29(6):3 17-322. 

5. Kaping D, Vinck M, Hutchison RM, Everling S, Womelsdorf T: Specific 
contributions of ventromedial, anterior cingulate, and lateral prefrontal 
cortex for attentional selection and stimulus valuation. PLoS Biol 201 1, 
9(12):el001224. 

6. Balcarras (VI, Ardid S, Kaping D, Everling S, Womelsdorf T: Learning of 
expected stimulus values in a selective attentional foraging task by 
nonhuman primates. 5oc Neurosci Abstr 201 2. 



doi:1 0.1 1 86/1 471-2202-1 5-S1 -P8 

Cite this article as: Ardid et al: "Adaptive learning" as a mechanistic 
candidate for reaching optimal task-set representations flexibly. BMC 

Neuroscience 201 4 15(Suppl 1):P8. 



Submit your next manuscript to BioMed Central 
and take full advantage of: 

• Convenient online submission 

• Thorough peer review 

• No space constraints or color figure charges 

• Immediate publication on acceptance 

• Inclusion in PubMed, CAS, Scopus and Google Scholar 

• Research which is freely available for redistribution 



Submit your manuscript at S~\ RioM _j r P ntral 

www.biomedcentral.com/submit \ J Blomea ^e™ 31 



